About | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange

How to train and use the BiGG object detector

Description: This tutorial presents a brief description of the steps needed to train and use the BiGG object detector

Tutorial Level: INTERMEDIATE

Training

The detector is trained by presenting it with a series of images of the object of interest (the object to be detected) where the object's segmentation is known. The segmentations must be published on a topic of type rein/MaskArray, each segmentation message synchronized with the corresponding image.

There are various ways of generating image segmentations to use for training, we have experimented with two ways:

• segmenting the point cloud above a table and projecting it into an image
• manually selecting a 3D bounding box around an object and tracking it using a checkerboard near the object

For each of the above cases there is a corresponding launch file inside the bigg_detector package (bigg_training_ptu.launch and bigg_training_cam.launch).

Using a manually selected 3D bounding box

When using a manually selected 3D bounding box, the process is the following:

roslaunch rein select3dobj.launch

This will launch a window showing the image from the camera. When the checkerboard is detected in the image, pressing the 'Enter' key will cause the image to freeze and allow the user to select the 3D wounding box around the object. A second 'Enter' key will confirm the selection and the node will start tracking the 3D bounding box and will also publish the masks representing the segmentation.

The training launch file can be then started:

roslaunch bigg_detector bigg_training_cam.launch

This will launch a simple GUI showing the segmented image. Pressing the 'Start training' button will begin the training process. For training, the object detector must be presented with images of the trained object from multiple different viewpoints. Pressing 'Save Model' will finish training and save the trained model.

Using a segmented point cloud

If a segmented point cloud of the object to be trained is available, it can be projected into the image to obtain the image segmentation, which can be used for training.

roslaunch bigg_detector bigg_training_ptu.launch

Using the object detector

The object detector can be started in detection mode with the following launch file:

roslaunch bigg_detector bigg_detection_cam.launch